{"cells":[{"cell_type":"markdown","metadata":{"id":"uh5rwbr4q5Nw"},"source":["## GenAI HW5: LLM Fine-tuning\n","In this homework, you will fine-tune your own LLM to enable the LLM to write Tang poems. For more details, please refer to [homework slides](https://docs.google.com/presentation/d/1NINc4Av4pJDMsGL1N_B4CUviwU13-H0RiCvs2K01O-I/edit#slide=id.g265f1963f2a_0_5)\n","\n","**TODOs**\n","1. Read the slides and make sure you know the objectives of this homework.\n","2. Save a copy of this Colab notebook.\n","3. Follow the steps in this Colab notebook to fine-tune your LLM.\n","4. Evaluate outputs using DaVinci Assistant (達哥批改助教)\n","5. Update results to NTU COOL\n","6. (optional) use [score parsing program](https://colab.research.google.com/drive/1OIuGKjmKghIgHtBlsWcGLcFP0pNQRNQE?usp=sharing) to check your score\n","\n","If you have any questions, please contact the TAs via TA hours, NTU COOL, or email to ntu-gen-ai-2024-spring-ta@googlegroups.com\n","\n","**Tips**: At the top of each cell, it shows whether you need to change hyperparameters in that cell and how long it might take."]},{"cell_type":"markdown","metadata":{"id":"eRKf5DA69b3r"},"source":["## Activate GPU\n","Since you will fine-tune a model, you **MUST** activate GPU so that this homework can be done in reasonable time (1~2 hrs)"]},{"cell_type":"markdown","source":["### **MUST READ**:\n","\n","Colab does **NOT** guarantee the GPU access for free user ([ref](https://research.google.com/colaboratory/faq.html#idle-timeouts)). It is possible you get an message saying \"Cannot connect to GPU backend\" which means there are no enough GPU resources for you now. When this happens, you may need to **wait for one (or more) day or login different Google account to do the homework**."],"metadata":{"id":"oYO3jZXvpv0c"}},{"cell_type":"markdown","metadata":{"id":"vfYdhzJy-H9e"},"source":["### Enable GPU\n","\n","1. Click on \"Runtime\" (or \"執行階段\") in the header.\n","2. Click on \"Change runtime type\" (or \"變更執行階段類型\") in the drop menu.\n","3. Select \"T4 GPU\" and save. (You can select \"A100 GPU\" or \"V100 GPU\" if you have Colab Pro)\n","![EnableGPU.png]()"]},{"cell_type":"markdown","metadata":{"id":"RrXWZefE-WXJ"},"source":["## Mount Google Drive\n","You **MUST** mount your Google Drive so that the  results can be saved on it."]},{"cell_type":"markdown","metadata":{"id":"SFKTMRc2-fs1"},"source":["To mount Google Drive, please follow these steps:\n","\n","1. Run the next cell.\n","2. Click on \"Connect to Google Drive\". (There should be a window popup)\n","3. Sign in your account and give permission."]},{"cell_type":"markdown","metadata":{"id":"J8cXGoOcCLrK"},"source":["The following code block takes about **1** minutes to run, but it may vary depending on the condition of Colab and Google Drive."]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":28345,"status":"ok","timestamp":1738913479915,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"},"user_tz":-480},"id":"4irqfznNAlrZ","outputId":"776c3bcf-c9d6-42da-8d87-12c7e2f69c85"},"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/drive\n"]}],"source":["from google.colab import drive\n","drive.mount('/content/drive')"]},{"cell_type":"code","source":["!ls"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ciuBr_AKZA8c","executionInfo":{"status":"ok","timestamp":1738913543450,"user_tz":-480,"elapsed":156,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"21ac6e5f-42e7-42ee-cc8a-7234df8158ef"},"execution_count":2,"outputs":[{"output_type":"stream","name":"stdout","text":["drive  sample_data\n"]}]},{"cell_type":"code","source":["import sys\n","print(sys.version)\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"x3Mq8bBhVAVc","executionInfo":{"status":"ok","timestamp":1738913670263,"user_tz":-480,"elapsed":9,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"779667b1-6f55-41cd-8f23-4a450255d874"},"execution_count":3,"outputs":[{"output_type":"stream","name":"stdout","text":["3.11.11 (main, Dec  4 2024, 08:55:07) [GCC 11.4.0]\n"]}]},{"cell_type":"markdown","metadata":{"id":"0hJFrdFQn84M"},"source":["## Install Packages\n","We install and import some well-written packages created by others to facilitate the fine-tuning process."]},{"cell_type":"code","source":["!pip install bitsandbytes\n","!pip install datasets\n","!pip install colorama\n","!pip install fsspec\n","!pip install peft==0.9.0"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"KeVbDF1aWBbX","executionInfo":{"status":"ok","timestamp":1738914102407,"user_tz":-480,"elapsed":22768,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"8b7045b7-1dde-4b0b-c604-1325ebb4c4cd"},"execution_count":1,"outputs":[{"output_type":"stream","name":"stdout","text":["Requirement already satisfied: bitsandbytes in /usr/local/lib/python3.11/dist-packages (0.45.2)\n","Requirement already satisfied: torch<3,>=2.0 in /usr/local/lib/python3.11/dist-packages (from bitsandbytes) (2.5.1+cu124)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.11/dist-packages (from bitsandbytes) (1.26.4)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (3.17.0)\n","Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (4.12.2)\n","Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (3.4.2)\n","Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (3.1.5)\n","Requirement already satisfied: fsspec in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (2024.9.0)\n","Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (12.4.127)\n","Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (12.4.127)\n","Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (12.4.127)\n","Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (9.1.0.70)\n","Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (12.4.5.8)\n","Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (11.2.1.3)\n","Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (10.3.5.147)\n","Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (11.6.1.9)\n","Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (12.3.1.170)\n","Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (2.21.5)\n","Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (12.4.127)\n","Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (12.4.127)\n","Requirement already satisfied: triton==3.1.0 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (3.1.0)\n","Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.11/dist-packages (from torch<3,>=2.0->bitsandbytes) (1.13.1)\n","Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy==1.13.1->torch<3,>=2.0->bitsandbytes) (1.3.0)\n","Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch<3,>=2.0->bitsandbytes) (3.0.2)\n","Requirement already satisfied: datasets in /usr/local/lib/python3.11/dist-packages (3.2.0)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from datasets) (3.17.0)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.11/dist-packages (from datasets) (1.26.4)\n","Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.11/dist-packages (from datasets) (17.0.0)\n","Requirement already satisfied: dill<0.3.9,>=0.3.0 in /usr/local/lib/python3.11/dist-packages (from datasets) (0.3.8)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.11/dist-packages (from datasets) (2.2.2)\n","Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.11/dist-packages (from datasets) (2.32.3)\n","Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.11/dist-packages (from datasets) (4.67.1)\n","Requirement already satisfied: xxhash in /usr/local/lib/python3.11/dist-packages (from datasets) (3.5.0)\n","Requirement already satisfied: multiprocess<0.70.17 in /usr/local/lib/python3.11/dist-packages (from datasets) (0.70.16)\n","Requirement already satisfied: fsspec<=2024.9.0,>=2023.1.0 in /usr/local/lib/python3.11/dist-packages (from fsspec[http]<=2024.9.0,>=2023.1.0->datasets) (2024.9.0)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.11/dist-packages (from datasets) (3.11.11)\n","Requirement already satisfied: huggingface-hub>=0.23.0 in /usr/local/lib/python3.11/dist-packages (from datasets) (0.28.1)\n","Requirement already satisfied: packaging in /usr/local/lib/python3.11/dist-packages (from datasets) (24.2)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.11/dist-packages (from datasets) (6.0.2)\n","Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets) (2.4.4)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets) (1.3.2)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets) (25.1.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets) (1.5.0)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets) (6.1.0)\n","Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets) (0.2.1)\n","Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp->datasets) (1.18.3)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.23.0->datasets) (4.12.2)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (3.4.1)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (3.10)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (2.3.0)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (2025.1.31)\n","Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.11/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.11/dist-packages (from pandas->datasets) (2025.1)\n","Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.11/dist-packages (from pandas->datasets) (2025.1)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.11/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.17.0)\n","Requirement already satisfied: colorama in /usr/local/lib/python3.11/dist-packages (0.4.6)\n","Requirement already satisfied: fsspec in /usr/local/lib/python3.11/dist-packages (2024.9.0)\n","Requirement already satisfied: peft==0.9.0 in /usr/local/lib/python3.11/dist-packages (0.9.0)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (1.26.4)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (24.2)\n","Requirement already satisfied: psutil in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (5.9.5)\n","Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (6.0.2)\n","Requirement already satisfied: torch>=1.13.0 in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (2.5.1+cu124)\n","Requirement already satisfied: transformers in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (4.48.2)\n","Requirement already satisfied: tqdm in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (4.67.1)\n","Requirement already satisfied: accelerate>=0.21.0 in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (1.3.0)\n","Requirement already satisfied: safetensors in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (0.5.2)\n","Requirement already satisfied: huggingface-hub>=0.17.0 in /usr/local/lib/python3.11/dist-packages (from peft==0.9.0) (0.28.1)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (3.17.0)\n","Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (2024.9.0)\n","Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (2.32.3)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (4.12.2)\n","Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (3.4.2)\n","Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (3.1.5)\n","Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (12.4.127)\n","Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (12.4.127)\n","Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (12.4.127)\n","Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (9.1.0.70)\n","Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (12.4.5.8)\n","Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (11.2.1.3)\n","Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (10.3.5.147)\n","Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (11.6.1.9)\n","Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (12.3.1.170)\n","Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (2.21.5)\n","Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (12.4.127)\n","Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (12.4.127)\n","Requirement already satisfied: triton==3.1.0 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (3.1.0)\n","Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.11/dist-packages (from torch>=1.13.0->peft==0.9.0) (1.13.1)\n","Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy==1.13.1->torch>=1.13.0->peft==0.9.0) (1.3.0)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.11/dist-packages (from transformers->peft==0.9.0) (2024.11.6)\n","Requirement already satisfied: tokenizers<0.22,>=0.21 in /usr/local/lib/python3.11/dist-packages (from transformers->peft==0.9.0) (0.21.0)\n","Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch>=1.13.0->peft==0.9.0) (3.0.2)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (3.4.1)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (3.10)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (2.3.0)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (2025.1.31)\n"]}]},{"cell_type":"markdown","metadata":{"id":"4UoAVpcAELzB"},"source":["The following code block takes about **5** minutes to run, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":3,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":35},"executionInfo":{"elapsed":9,"status":"ok","timestamp":1738914154092,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"},"user_tz":-480},"id":"kuRMjk0rtWBx","outputId":"35383657-3ebd-4ab5-ae0e-0072f86b8956"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["' It is recommmended NOT to change codes in this cell '"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":3}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","# !pip install bitsandbytes==0.43.0\n","# !pip install datasets==2.10.1\n","# !pip install transformers==4.38.2\n","# !pip install peft==0.9.0\n","# !pip install sentencepiece==0.1.99\n","# !pip install -U accelerate==0.28.0\n","# !pip install colorama==0.4.6\n","# !pip install fsspec==2023.9.2"]},{"cell_type":"markdown","metadata":{"id":"3OMmuIblEXiU"},"source":["The following code block takes about **20** seconds to run, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":2,"metadata":{"id":"ZVVG_SQrvFpe","executionInfo":{"status":"ok","timestamp":1738914128885,"user_tz":-480,"elapsed":21049,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","import os\n","import sys\n","import argparse\n","import json\n","import warnings\n","import logging\n","warnings.filterwarnings(\"ignore\")\n","\n","import torch\n","import torch.nn as nn\n","import bitsandbytes as bnb\n","from datasets import load_dataset, load_from_disk\n","import transformers, datasets\n","from peft import PeftModel\n","from colorama import *\n","\n","from tqdm import tqdm\n","from transformers import AutoTokenizer, AutoConfig, AutoModelForCausalLM, BitsAndBytesConfig\n","from transformers import GenerationConfig\n","from peft import (\n","    prepare_model_for_int8_training,\n","    LoraConfig,\n","    get_peft_model,\n","    get_peft_model_state_dict,\n","    prepare_model_for_kbit_training\n",")"]},{"cell_type":"code","source":["!nvidia-smi"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"emEu81RIdrtV","executionInfo":{"status":"ok","timestamp":1738748122398,"user_tz":-480,"elapsed":4,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"375cebe9-18d0-470f-cc14-4510f57d3c79"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Wed Feb  5 09:35:22 2025       \n","+-----------------------------------------------------------------------------------------+\n","| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |\n","|-----------------------------------------+------------------------+----------------------+\n","| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |\n","| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |\n","|                                         |                        |               MIG M. |\n","|=========================================+========================+======================|\n","|   0  Tesla T4                       Off |   00000000:00:04.0 Off |                    0 |\n","| N/A   43C    P8              9W /   70W |       2MiB /  15360MiB |      0%      Default |\n","|                                         |                        |                  N/A |\n","+-----------------------------------------+------------------------+----------------------+\n","                                                                                         \n","+-----------------------------------------------------------------------------------------+\n","| Processes:                                                                              |\n","|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |\n","|        ID   ID                                                               Usage      |\n","|=========================================================================================|\n","|  No running processes found                                                             |\n","+-----------------------------------------------------------------------------------------+\n"]}]},{"cell_type":"code","source":["import torch\n","print(torch.version.cuda)\n","print(torch.__version__)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Y7c-5NgceC4l","executionInfo":{"status":"ok","timestamp":1738914164006,"user_tz":-480,"elapsed":5,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"61300737-e07a-45ed-a8b2-40452417692a"},"execution_count":4,"outputs":[{"output_type":"stream","name":"stdout","text":["12.4\n","2.5.1+cu124\n"]}]},{"cell_type":"markdown","metadata":{"id":"dCo1znQhpBdt"},"source":["## Download Dataset for Fine-tuning"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1721,"status":"ok","timestamp":1738914170876,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"},"user_tz":-480},"id":"7AI9jvvGtav2","outputId":"dd7ce7fd-b6e8-4e80-d409-5881d611d0f0"},"outputs":[{"output_type":"stream","name":"stdout","text":["Cloning into 'GenAI-Hw5'...\n","remote: Enumerating objects: 38, done.\u001b[K\n","remote: Counting objects: 100% (38/38), done.\u001b[K\n","remote: Compressing objects: 100% (29/29), done.\u001b[K\n","remote: Total 38 (delta 15), reused 26 (delta 7), pack-reused 0 (from 0)\u001b[K\n","Receiving objects: 100% (38/38), 3.68 MiB | 7.36 MiB/s, done.\n","Resolving deltas: 100% (15/15), done.\n"]}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","# Download Training dataset\n","# reference:https://github.com/chinese-poetry/chinese-poetry/tree/master/%E5%85%A8%E5%94%90%E8%AF%97?fbclid=IwAR2bM14S42T-VtrvMi3wywCqKfYJraBtMl7QVTo0qyPMjX9jj9Vj3JepFBA\n","!git clone https://github.com/CheeEn-Yu/GenAI-Hw5.git"]},{"cell_type":"markdown","metadata":{"id":"ESzwsaT9poC4"},"source":["## Fix Random Seeds\n","There may be some randomness involved in the fine-tuning process. We fix random seeds to make the result reproducible."]},{"cell_type":"code","execution_count":6,"metadata":{"id":"37Ra-OARteHI","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1738914198908,"user_tz":-480,"elapsed":8,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"55e10919-9459-4e9a-b6c1-621355d0680f"},"outputs":[{"output_type":"stream","name":"stdout","text":["yes available\n"]}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","seed = 42\n","torch.backends.cudnn.deterministic = True\n","torch.backends.cudnn.benchmark = False\n","torch.manual_seed(seed)\n","if torch.cuda.is_available():\n","    torch.cuda.manual_seed_all(seed)\n","    print(\"yes available\")"]},{"cell_type":"markdown","metadata":{"id":"jTaVpMgzp3oC"},"source":["## Define Some Useful Functions"]},{"cell_type":"code","execution_count":7,"metadata":{"id":"dKjoLO3xtfM1","executionInfo":{"status":"ok","timestamp":1738914521950,"user_tz":-480,"elapsed":15,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","# 生成訓練資料\n","def generate_training_data(data_point):\n","    \"\"\"\n","    (1) Goal: 将输入和输出文本转换为模型可读取的 tokens\n","        - This function is used to transform a data point (input and output texts) to tokens that our model can read\n","\n","    (2) Arguments:\n","        - data_point: dict, with field \"instruction\", \"input\", and \"output\" which are all str\n","\n","    (3) Returns:\n","        - a dict with model's input tokens, attention mask that make our model causal, and corresponding output targets\n","\n","    (3) Example:\n","        - If you construct a dict, data_point_1, with field \"instruction\", \"input\", and \"output\" which are all str, you can use the function like this:\n","            formulate_article(data_point_1)\n","\n","    \"\"\"\n","    # construct full input prompt 构建完整的输入提示词\n","    prompt = f\"\"\"\\\n","[INST] <<SYS>>\n","You are a helpful assistant and good at writing Tang poem. 你是一個樂於助人的助手且擅長寫唐詩。\n","<</SYS>>\n","\n","{data_point[\"instruction\"]}\n","{data_point[\"input\"]}\n","[/INST]\"\"\"\n","    # count the number of input tokens 计算用户提示词的 token 数量\n","    len_user_prompt_tokens = (\n","        len(\n","            tokenizer(\n","                prompt,\n","                truncation=True,\n","                max_length=CUTOFF_LEN + 1,\n","                padding=\"max_length\",\n","            )[\"input_ids\"]\n","        ) - 1\n","    )\n","    # transform input prompt into tokens 将完整的输入和输出转换为 tokens\n","    full_tokens = tokenizer(\n","        prompt + \" \" + data_point[\"output\"] + \"</s>\",\n","        truncation=True,\n","        max_length=CUTOFF_LEN + 1,\n","        padding=\"max_length\",\n","    )[\"input_ids\"][:-1]\n","    return {\n","        \"input_ids\": full_tokens,\n","        \"labels\": [-100] * len_user_prompt_tokens\n","        + full_tokens[len_user_prompt_tokens:],\n","        \"attention_mask\": [1] * (len(full_tokens)),\n","    }\n","\n","# 進行生成回覆的評估\n","def evaluate(instruction, generation_config, max_len, input=\"\", verbose=True):\n","    \"\"\"\n","    (1) Goal:\n","        - This function is used to get the model's output given input strings 获取模型在给定输入下的生成结果\n","\n","    (2) Arguments:\n","        - instruction: str, description of what you want model to do\n","        - generation_config: transformers.GenerationConfig object, to specify decoding parameters relating to model inference\n","        - max_len: int, max length of model's output\n","        - input: str, input string the model needs to solve the instruction, default is \"\" (no input)\n","        - verbose: bool, whether to print the mode's output, default is True\n","\n","    (3) Returns:\n","        - output: str, the mode's response according to the instruction and the input\n","\n","    (3) Example:\n","        - If you the instruction is \"ABC\" and the input is \"DEF\" and you want model to give an answer under 128 tokens, you can use the function like this:\n","            evaluate(instruction=\"ABC\", generation_config=generation_config, max_len=128, input=\"DEF\")\n","\n","    \"\"\"\n","    # construct full input prompt\n","    prompt = f\"\"\"\\\n","[INST] <<SYS>>\n","You are a helpful assistant and good at writing Tang poem. 你是一個樂於助人的助手且擅長寫唐詩。\n","<</SYS>>\n","\n","{instruction}\n","{input}\n","[/INST]\"\"\"\n","    # 將提示文本轉換為模型所需的數字表示形式\n","    inputs = tokenizer(prompt, return_tensors=\"pt\")\n","    input_ids = inputs[\"input_ids\"].cuda()\n","    # 使用模型進行生成回覆\n","    generation_output = model.generate(\n","        input_ids=input_ids,\n","        generation_config=generation_config,\n","        return_dict_in_generate=True,\n","        output_scores=True,\n","        max_new_tokens=max_len,\n","    )\n","    # 將生成的回覆解碼並印出\n","    for s in generation_output.sequences:\n","        output = tokenizer.decode(s)\n","        output = output.split(\"[/INST]\")[1].replace(\"</s>\", \"\").replace(\"<s>\", \"\").replace(\"Assistant:\", \"\").replace(\"Assistant\", \"\").strip()\n","        if (verbose):\n","            print(output)\n","\n","    return output\n"]},{"cell_type":"markdown","metadata":{"id":"SNxuuclCqFf5"},"source":["## Download model and inference before fine-tuning"]},{"cell_type":"markdown","source":["The following code block takes about **10** minutes to run if you use the default setting, but it may vary depending on the condition of Colab."],"metadata":{"id":"qmfEFM7TNuRC"}},{"cell_type":"code","source":["\"\"\" You may want (but not necessarily need) to change the LLM model \"\"\"\n","\n","# model_name = \"/content/TAIDE-LX-7B-Chat\"                            # 設定想要用來進行fine-tune的模型，預設是使用TAIDE 7B的模型\n","model_name = \"MediaTek-Research/Breeze-7B-Instruct-v0_1\"   # 若想選擇使用MediaTek Breeze 7B的模型，可以將這行最前面的 \"#\" 刪除，並把底下 \"!\" 開頭的兩行刪除\n","\n","# If you want to use the TAIDE model, you should check out the TAIDE L Models Community License Agreement (https://drive.google.com/file/d/1FcUZjbUH6jr4xoCyAronN_slLgcdhEUd/view) first.\n","# Once you use it, it means you agree to the terms of the agreement.\n","# !wget -O taide_7b.zip \"https://www.dropbox.com/scl/fi/harnetdwx2ttq1xt94rin/TAIDE-LX-7B-Chat.zip?rlkey=yzyf5nxztw6farpwyyildx5s3&st=s22mz5ao&dl=0\"\n","\n","# !unzip taide_7b.zip"],"metadata":{"id":"stY76riONxSz","executionInfo":{"status":"ok","timestamp":1738914696257,"user_tz":-480,"elapsed":5,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"execution_count":10,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"DFq6WXaBnxYq"},"source":["## Inference before Fine-tuning\n","Let's first see what our model can do without fine-tuning."]},{"cell_type":"markdown","metadata":{"id":"ox5QRpz4NiPg"},"source":["The following code block takes about **2** minutes to run if you use the default setting, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":11,"metadata":{"id":"e1sFQbHGn3Bw","colab":{"base_uri":"https://localhost:8080/","height":401,"referenced_widgets":["bcfd3dc154284e4b965b8bc67ff78cec","29bbe0b236f84b2cba1855d7588e79d1","24e7258b92454dc9827d2957d5fca17e","49139cb882484ef2a4ab90cd3ad8b57d","0891c210b7c94702a9b59a8405d238f0","55fdaa9a899f4d4393b6165d0e46ff1e","c654b9c398124d63a45f618113f05217","8b198efb30f8497194b01f000097b492","6a4154e2b36241e98796d55f8e95f032","4029dc88032d4c6f9a71acf1da7d8da2","a70623b2fdaa4266b636262801e11d2c","820b125bec9044b5a33378e0269261ec","23b0b3904ecc4220844926e574ab112d","1096aa12633b4bd08dbe83932cc8a757","f707fdf45ba846fab2f912d082d536b1","109991a5f4464902bf868d3921cdc2b9","2d4c62aacff84ef38c962fe528dfca03","cfce5e6ff1a54e9081201ae6d1abe90b","82186e29d0704b98a62f67f08c55e862","6911fda9a1434b8697b0840a01dbe4b0","82b4f6a98ace423a9fee5183293a39e5","340932d9058d4897a41026023402b8e1","bce2c017697d4c2480d28ab758852d2c","cacab48531b24d549da3c76e1f526687","d7cb2badde8548e394127b5554dc24d9","4df6ff5f4d094ce29d00047ef0777635","cf0d138e965a4bc8b2dc66affda9c98c","0edbdd84974242a3b7887b98f8457b9a","c0a47a3d93db4a97ab179ad4069a831f","46d034094cbd4484a1664e2ad6877457","3dd9c00a6b32462bb4653357f9fc5eb7","3819b9d87fa147be83e1c7b64d8418ca","c1ef0cddbcec45f58fedd26807a03567","666066284fc04acfb3f1ffc66f78cc1d","28007f024c5742aabb0a9bc1f6640662","d1de74fcdbff45dda9c5024d63df87b6","4305114af98e45a9ae9c8d88866ea7c7","3240c32f8b40481a8150d3b7fb05cd95","c8a87b0c75a14436887d927df61409a4","14a84072c51243ceb1162ee19a796ef3","c6b5cea33d5c40fc975ab835a7ef83bc","e29de32052a24575a107c9f57df2fca5","004df69a0ab247649ba99ef586f20741","1ae50934a52849da845a77da7eb3e4e2","76bd16da496c4f57b85c6a369869fa3b","c03520becbf5402fb1e6e2aab6bad158","0d27882a68bc468c946c61e803f12001","ee39c60ad86d4d50b27656ddd63bfd22","140b00a74bea4582a20e407107da6251","d401a7e1f4b8427f85ff2ad1f85f4143","55d6e9b0d1af439abc98002f45b1c8c1","a4cd4d55813343fa90567018063023ae","2e36e1bd3b8c49ee9f050ac5cc04397f","8eb769686d24421185b02eb627d3788e","a9e9a511989f41d1bef73532f67d1457","3c5c70522e77433796cefc9e2c13953e","3f1e3a686adb401288f09689de544a3b","ebaaac57fac34b2293cad1e4bc5da112","b3cc0a7bfbb74d5489a2b5352df702dc","73dc368955864dbe8a704188c06bbc4b","e42360c06b424aa9badab184d62dd451","7668843d095943ad862509f0cc690bc7","addf29833b5d4b7790a688a42a073d50","478e77ba0f9c4b2da2c847440df7105c","623e941b653d48f9bf8ed1fe3e17fb12","cdf9c8d0abf34e26b83346fe5d844ce5","f238d91216ef49808d854faa5a3de369","ec1e6656ee174169ac12b33bca02a8e5","2b02c83f74a24bcfbc241f0e4eede897","f627283c91704f7295f342ea87b6c018","0a0120ff6ad14d0f9927d9e9162a9afc","6351ee9201104df7b594c30f1813dc8e","68c7d5856c3544f29d5a79870fef6d40","21074198381245af9fe93a519946b411","22b961876ded4e638d600661973fcf4d","f784ff58fc4c4d9a963296fa0bab036b","1823b59a47264480b5e897e789930b42","99d2dde3fcf345118b1a9731c5ccd04d","d36b4697da1d4f61b9e30a7cf3aaf0e9","f0f72eb4c633433b9c2f6a5b4183f497","aad2a2577b624a0480467cf110230162","2d9ac1a648a34f7b8f94b444b2a04ffd","7f8c08bf7693425dbea463c363ce5973","e0ef488369ea4e7d993e3ff46816225e","e772972c082a45bba49dadc7f6296481","8b103e043b6d43cf917b244b625d05b6","1bd01d619b1e400d96f5ffdc89c6c72d","27155d4f1cb4473c954096455d7ee7b6","52bf15bf4d4f47e4acad99cd0457dd35","1900cba726d64d3fb527191fe4e22b2f","74d8c2d0f13e4388bb188f0803db840f","428cfa7372af4f8ca4ce33303a0847c8","d6fb6a4af4ce4b55adf75e9eaf31aef3","557f2b17f9fc49ad89cc15474976b99e","d50e8cbc367c46a293907b36f0b3732c","6e5a02789b3f47949654e26890e22422","9f19625acd984d6d912a92f1a91b3c89","af2ff52b55ec42daa8116b5d4d02a672","f6c0fb2e19534f3287851932defaa4b3","04d073d67706463ca88a81fbe87e77f0","7c8f02283f4149e5bb58b8ad16d93ac4","42d4c57ffc3b4d5a84b1971cec46cdf9","9ed625ec2d5345a98639b040d84dfa2b","e0793957ac1047538ce377c84bea1674","927530148342418d88c2793afd4b5e46","6687d3d94327447cb8f8cc85dd8b63d8","479428fa05214e4c9dc6a86088fbee12","526470a79ce248c8861201ca0fb6e6e5","7758977be2cf47b4869c24fa8ae59b4e","ce2cc8aeeece4bd59a84d730f962321f","eb5c8ceab7984b05a14337dfdb4be86d","fa77618315274b93bbc35c99e61ff262","5cd0801333334acd963d3831745317dc","51f2b2ec63674da9ad545ae353db20f2","b8b416066fbe48a3a0cd47062fdee18f","c5f2e03f86244d74829ca03d0cbaeabb","8a7ff402838e44929b2d07f88f88226a","f4f8559486bf49ac865e1c89b23fac96","1bf1eac05a3b48a49f6c01ca2471dd68","2738c81c8c334e28b126843c5b6a9c2f","828d2fa937f742638757378393ae9557","34015942bc704bdcb4076d2132fcc0f6","d704822cb44940b89fc1f1054969ec17","c5d30f3ea5f44171855ffed11ccd3d12","85701f001be740e5b0c3390a375caa0e","e6b9f589d4a8492a9ad4e2967afe5b33","5748406df7434a6e98e2e48d23a918de","40a72a290f9540048425f78637775a91","e62909803e184e938879d1fa77ff7fab","b70acf67d7da46f2b6d32b72b31c7d87","1587793d20e94d3a83f2f2db74fe91a4","f8a3582a5ee64fa291592fee5e6a8a41"]},"executionInfo":{"status":"ok","timestamp":1738915136820,"user_tz":-480,"elapsed":432691,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"ebc9e3d5-2ff7-4c42-df97-c55a228659fb"},"outputs":[{"output_type":"display_data","data":{"text/plain":["config.json:   0%|          | 0.00/618 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"bcfd3dc154284e4b965b8bc67ff78cec"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["model.safetensors.index.json:   0%|          | 0.00/25.1k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"820b125bec9044b5a33378e0269261ec"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"bce2c017697d4c2480d28ab758852d2c"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["model-00001-of-00002.safetensors:   0%|          | 0.00/9.99G [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"666066284fc04acfb3f1ffc66f78cc1d"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["model-00002-of-00002.safetensors:   0%|          | 0.00/4.99G [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"76bd16da496c4f57b85c6a369869fa3b"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3c5c70522e77433796cefc9e2c13953e"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["generation_config.json:   0%|          | 0.00/111 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"f238d91216ef49808d854faa5a3de369"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["tokenizer_config.json:   0%|          | 0.00/2.29k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"99d2dde3fcf345118b1a9731c5ccd04d"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["tokenizer.model:   0%|          | 0.00/911k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"52bf15bf4d4f47e4acad99cd0457dd35"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["tokenizer.json:   0%|          | 0.00/2.79M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"04d073d67706463ca88a81fbe87e77f0"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["added_tokens.json:   0%|          | 0.00/39.0 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"eb5c8ceab7984b05a14337dfdb4be86d"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["special_tokens_map.json:   0%|          | 0.00/551 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"34015942bc704bdcb4076d2132fcc0f6"}},"metadata":{}}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","cache_dir = \"./cache\"\n","\n","nf4_config = BitsAndBytesConfig(\n","   load_in_4bit=True,\n","   bnb_4bit_quant_type=\"nf4\",\n","   bnb_4bit_use_double_quant=True,\n","   bnb_4bit_compute_dtype=torch.bfloat16\n",")\n","\n","# 從指定的模型名稱或路徑載入預訓練的語言模型\n","model = AutoModelForCausalLM.from_pretrained(\n","    model_name,\n","    cache_dir=cache_dir,\n","    quantization_config=nf4_config,\n","    low_cpu_mem_usage = True\n",")\n","\n","# 創建 tokenizer 並設定結束符號 (eos_token)\n","logging.getLogger('transformers').setLevel(logging.ERROR)\n","tokenizer = AutoTokenizer.from_pretrained(\n","    model_name,\n","    add_eos_token=True,\n","    cache_dir=cache_dir,\n","    quantization_config=nf4_config\n",")\n","tokenizer.pad_token = tokenizer.eos_token\n","\n","# 設定模型推理時需要用到的decoding parameters\n","max_len = 128\n","generation_config = GenerationConfig(\n","    do_sample=True,\n","    temperature=0.1,\n","    num_beams=1,\n","    top_p=0.3,\n","    no_repeat_ngram_size=3,\n","    pad_token_id=2,\n",")"]},{"cell_type":"markdown","metadata":{"id":"0LyTYIcDOAkO"},"source":["The following code block takes about **1** minutes to run if you use the default setting, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":12,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"QJyuoPoO2TCr","outputId":"895f6f1b-bd2e-47a3-bfb9-beec8eade450","executionInfo":{"status":"ok","timestamp":1738915331099,"user_tz":-480,"elapsed":7303,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["Example 1:\n","模型輸入:\n","以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。相見時難別亦難，東風無力百花殘。\n","\n","模型輸出:\n","相會難，別亦艱難，東風吹，百花落。\n","--------------------------------------------------------------------------------\n","Example 2:\n","模型輸入:\n","以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。重帷深下莫愁堂，臥後清宵細細長。\n","\n","模型輸出:\n","重帷暗下無愁影，臥后清宵細長長。\n","--------------------------------------------------------------------------------\n","Example 3:\n","模型輸入:\n","以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。芳辰追逸趣，禁苑信多奇。\n","\n","模型輸出:\n","芳辰追逐趣，禁止苑中奇。\n","--------------------------------------------------------------------------------\n"]}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","# demo examples\n","test_tang_list = ['相見時難別亦難，東風無力百花殘。', '重帷深下莫愁堂，臥後清宵細細長。', '芳辰追逸趣，禁苑信多奇。']\n","\n","# get the model output for each examples\n","demo_before_finetune = []\n","for tang in test_tang_list:\n","  demo_before_finetune.append(f'模型輸入:\\n以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。{tang}\\n\\n模型輸出:\\n'+evaluate('以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。', generation_config, max_len, tang, verbose = False))\n","\n","# print and store the output to text file\n","for idx in range(len(demo_before_finetune)):\n","  print(f\"Example {idx + 1}:\")\n","  print(demo_before_finetune[idx])\n","  print(\"-\" * 80)\n"]},{"cell_type":"markdown","metadata":{"id":"Stf_U-9FqPjZ"},"source":["## Set Hyperarameters for Fine-tuning\n","\n"]},{"cell_type":"code","source":["\"\"\" It is highly recommended you try to play around this hyperparameter \"\"\"\n","\n","num_train_data = 1040 # 設定用來訓練的資料數量，可設置的最大值為5000。在大部分情況下會希望訓練資料盡量越多越好，這會讓模型看過更多樣化的詩句，進而提升生成品質，但是也會增加訓練的時間\n","                      # 使用預設參數(1040): fine-tuning大約需要25分鐘，完整跑完所有cell大約需要50分鐘\n","                      # 使用最大值(5000): fine-tuning大約需要100分鐘，完整跑完所有cell大約需要120分鐘"],"metadata":{"id":"Q2ilhBGhTDtU","executionInfo":{"status":"ok","timestamp":1738915406922,"user_tz":-480,"elapsed":66,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"execution_count":13,"outputs":[]},{"cell_type":"code","execution_count":14,"metadata":{"id":"c1fbknrxXu3C","executionInfo":{"status":"ok","timestamp":1738915429499,"user_tz":-480,"elapsed":2,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[],"source":["\"\"\" You may want (but not necessarily need) to change some of these hyperparameters \"\"\"\n","\n","output_dir = \"/content/drive/MyDrive\"  # 設定作業結果輸出目錄 (如果想要把作業結果存在其他目錄底下可以修改這裡，強烈建議存在預設值的子目錄下，也就是Google Drive裡)\n","ckpt_dir = \"./exp1\" # 設定model checkpoint儲存目錄 (如果想要將model checkpoints存在其他目錄下可以修改這裡)\n","num_epoch = 1  # 設定訓練的總Epoch數 (數字越高，訓練越久，若使用免費版的colab需要注意訓練太久可能會斷線)\n","LEARNING_RATE = 3e-4  # 設定學習率"]},{"cell_type":"code","execution_count":15,"metadata":{"id":"oorrZOKpthsm","executionInfo":{"status":"ok","timestamp":1738915493187,"user_tz":-480,"elapsed":2,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","cache_dir = \"./cache\"  # 設定快取目錄路徑\n","from_ckpt = False  # 是否從checkpoint載入模型的權重，預設為否\n","ckpt_name = None  # 從特定checkpoint載入權重時使用的檔案名稱，預設為無\n","dataset_dir = \"./GenAI-Hw5/Tang_training_data.json\"  # 設定資料集的目錄或檔案路徑\n","logging_steps = 20  # 定義訓練過程中每隔多少步驟輸出一次訓練誌\n","save_steps = 65  # 定義訓練過程中每隔多少步驟保存一次模型\n","save_total_limit = 3  # 控制最多保留幾個模型checkpoint\n","report_to = None  # 設定上報實驗指標的目標，預設為無\n","MICRO_BATCH_SIZE = 4  # 定義微批次的大小\n","BATCH_SIZE = 16  # 定義一個批次的大小\n","GRADIENT_ACCUMULATION_STEPS = BATCH_SIZE // MICRO_BATCH_SIZE  # 計算每個微批次累積的梯度步數\n","CUTOFF_LEN = 256  # 設定文本截斷的最大長度\n","LORA_R = 8  # 設定LORA（Layer-wise Random Attention）的R值\n","LORA_ALPHA = 16  # 設定LORA的Alpha值\n","LORA_DROPOUT = 0.05  # 設定LORA的Dropout率\n","VAL_SET_SIZE = 0  # 設定驗證集的大小，預設為無\n","TARGET_MODULES = [\"q_proj\", \"up_proj\", \"o_proj\", \"k_proj\", \"down_proj\", \"gate_proj\", \"v_proj\"] # 設定目標模組，這些模組的權重將被保存為checkpoint\n","device_map = \"auto\"  # 設定設備映射，預設為\"auto\"\n","world_size = int(os.environ.get(\"WORLD_SIZE\", 1))  # 獲取環境變數\"WORLD_SIZE\"的值，若未設定則預設為1\n","ddp = world_size != 1  # 根據world_size判斷是否使用分散式數據處理(DDP)，若world_size為1則不使用DDP\n","if ddp:\n","    device_map = {\"\": int(os.environ.get(\"LOCAL_RANK\") or 0)}\n","    GRADIENT_ACCUMULATION_STEPS = GRADIENT_ACCUMULATION_STEPS // world_size"]},{"cell_type":"markdown","source":["## Start Fine-tuning"],"metadata":{"id":"REMmOD6L4tp9"}},{"cell_type":"markdown","metadata":{"id":"RkOt2hDhR4lG"},"source":["The following code block takes about **25** minutes to run if you use the default setting, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":16,"metadata":{"colab":{"referenced_widgets":["701997d9abf6402299b355c8a3fe8623","956ae37532b241a188a9aba7f87b7d46","3827589f4d68440f8e164fc234a6b0be","6dfc9dc0c0e244afb7690d3f8a34c10e","ce798caee48846099b6e9086ffdbb104","ce7f76813e24443ebcd18a7cf828d95b","7cddd41aa9de482b995f3355a30e8d20","1e49c31e57f8427c882f1d4435c497ff","abf06b78f9b5419a99007a3b755a99bc","f5800fd73dd84b46b62ad12f64f01824","137854b9b7334959863b836b071e0e98","f5d4abbaec95435d9d52f452b3465b39","abe48faf2fe44cd1a1844d27e1e81fde","003cec5011584bf38d4eb4670c57c0e4","18c7781c59064d1ebed245cab21c256f","cb3dd64ff5cb4bdf98c333f27614f843","d638472d17c44b258a1572ce965ceec1","2dce2f8cf3ca4d3fabb559da4aa52bad","8bcef1ca5bc44d98bf9ee1518f4a8d3f","73b914a2990b46d2aa207fdd7c1aceb8","22f94dfc5ab4461fa43c5b6f54be7be9","f68ac26de2404b43b3f11e4055345780","53a85e86099045978c28eee131501e43","adb03ec94e334ab69e320168cdd8673c","bff036d428c74bf09f7953d65a5ecd16","78eed1ace80746e7b9cf06e75c4e56d4","b661478f72734a649db8779b04dc84a0","bcdfb7a75f5d454985ef2ed43590b0f6","3845bcd464b84a9885eabae4e3b06b76","536b0eaa47cb4b6084eba8058da85014","77e8d7ee81134623829b50faa88b6150","b7bfa561b03d44f1b2258f183550f00a","0de6e2af6120413dacb838d007611c96"],"base_uri":"https://localhost:8080/","height":514},"id":"6W-xe7h9ti0x","outputId":"6d2df55f-3eb8-4f17-daba-c8617252c047","executionInfo":{"status":"ok","timestamp":1738917044435,"user_tz":-480,"elapsed":1398359,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[{"output_type":"display_data","data":{"text/plain":["Generating train split: 0 examples [00:00, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"701997d9abf6402299b355c8a3fe8623"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["Map:   0%|          | 0/1040 [00:00<?, ? examples/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"f5d4abbaec95435d9d52f452b3465b39"}},"metadata":{}},{"output_type":"stream","name":"stderr","text":["\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.\n"]},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["\n","        window._wandbApiKey = new Promise((resolve, reject) => {\n","            function loadScript(url) {\n","            return new Promise(function(resolve, reject) {\n","                let newScript = document.createElement(\"script\");\n","                newScript.onerror = reject;\n","                newScript.onload = resolve;\n","                document.body.appendChild(newScript);\n","                newScript.src = url;\n","            });\n","            }\n","            loadScript(\"https://cdn.jsdelivr.net/npm/postmate/build/postmate.min.js\").then(() => {\n","            const iframe = document.createElement('iframe')\n","            iframe.style.cssText = \"width:0;height:0;border:none\"\n","            document.body.appendChild(iframe)\n","            const handshake = new Postmate({\n","                container: iframe,\n","                url: 'https://wandb.ai/authorize'\n","            });\n","            const timeout = setTimeout(() => reject(\"Couldn't auto authenticate\"), 5000)\n","            handshake.then(function(child) {\n","                child.on('authorize', data => {\n","                    clearTimeout(timeout)\n","                    resolve(data)\n","                });\n","            });\n","            })\n","        });\n","    "]},"metadata":{}},{"output_type":"stream","name":"stderr","text":["\u001b[34m\u001b[1mwandb\u001b[0m: Logging into wandb.ai. (Learn how to deploy a W&B server locally: https://wandb.me/wandb-server)\n","\u001b[34m\u001b[1mwandb\u001b[0m: You can find your API key in your browser here: https://wandb.ai/authorize\n","wandb: Paste an API key from your profile and hit enter, or press ctrl+c to quit:"]},{"name":"stdout","output_type":"stream","text":[" ··········\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m If you're specifying your api key in code, ensure this code is not shared publicly.\n","\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m Consider setting the WANDB_API_KEY environment variable, or running `wandb login` from the command line.\n","\u001b[34m\u001b[1mwandb\u001b[0m: Appending key for api.wandb.ai to your netrc file: /root/.netrc\n","\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mjackymcy\u001b[0m (\u001b[33mjackymcy-mcy\u001b[0m) to \u001b[32mhttps://api.wandb.ai\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n","\u001b[34m\u001b[1mwandb\u001b[0m: Using wandb-core as the SDK backend.  Please refer to https://wandb.me/wandb-core for more information.\n"]},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["Tracking run with wandb version 0.19.5"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["Run data is saved locally in <code>/content/wandb/run-20250207_081147-2gxk502r</code>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["Syncing run <strong><a href='https://wandb.ai/jackymcy-mcy/huggingface/runs/2gxk502r' target=\"_blank\">./exp1</a></strong> to <a href='https://wandb.ai/jackymcy-mcy/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":[" View project at <a href='https://wandb.ai/jackymcy-mcy/huggingface' target=\"_blank\">https://wandb.ai/jackymcy-mcy/huggingface</a>"]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":[" View run at <a href='https://wandb.ai/jackymcy-mcy/huggingface/runs/2gxk502r' target=\"_blank\">https://wandb.ai/jackymcy-mcy/huggingface/runs/2gxk502r</a>"]},"metadata":{}},{"output_type":"stream","name":"stdout","text":["{'loss': 3.2524, 'grad_norm': 2.1677393913269043, 'learning_rate': 0.00011399999999999999, 'epoch': 0.3076923076923077}\n","{'loss': 2.0808, 'grad_norm': 1.8100664615631104, 'learning_rate': 0.000234, 'epoch': 0.6153846153846154}\n","{'loss': 1.9798, 'grad_norm': 1.383529782295227, 'learning_rate': 0.00011999999999999999, 'epoch': 0.9230769230769231}\n"]},{"output_type":"display_data","data":{"text/plain":["config.json:   0%|          | 0.00/618 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"53a85e86099045978c28eee131501e43"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["{'train_runtime': 1387.7601, 'train_samples_per_second': 0.749, 'train_steps_per_second': 0.047, 'train_loss': 2.399632131136381, 'epoch': 1.0}\n","\n"," If there's a warning about missing keys above, please disregard :)\n"]}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","# create the output directory you specify\n","os.makedirs(output_dir, exist_ok = True)\n","os.makedirs(ckpt_dir, exist_ok = True)\n","\n","# 根據 from_ckpt 標誌，從 checkpoint 載入模型權重\n","if from_ckpt:\n","    model = PeftModel.from_pretrained(model, ckpt_name)\n","\n","# 將模型準備好以使用 INT8 訓練\n","model = prepare_model_for_int8_training(model)\n","\n","# 使用 LoraConfig 配置 LORA 模型\n","config = LoraConfig(\n","    r=LORA_R,\n","    lora_alpha=LORA_ALPHA,\n","    target_modules=TARGET_MODULES,\n","    lora_dropout=LORA_DROPOUT,\n","    bias=\"none\",\n","    task_type=\"CAUSAL_LM\",\n",")\n","model = get_peft_model(model, config)\n","\n","# 將 tokenizer 的 padding token 設定為 0\n","tokenizer.pad_token_id = 0\n","\n","# 載入並處理訓練數據\n","with open(dataset_dir, \"r\", encoding = \"utf-8\") as f:\n","    data_json = json.load(f)\n","with open(\"tmp_dataset.json\", \"w\", encoding = \"utf-8\") as f:\n","    json.dump(data_json[:num_train_data], f, indent = 2, ensure_ascii = False)\n","\n","data = load_dataset('json', data_files=\"tmp_dataset.json\", download_mode=\"force_redownload\")\n","\n","# 將訓練數據分為訓練集和驗證集（若 VAL_SET_SIZE 大於 0）\n","if VAL_SET_SIZE > 0:\n","    train_val = data[\"train\"].train_test_split(\n","        test_size=VAL_SET_SIZE, shuffle=True, seed=42\n","    )\n","    train_data = train_val[\"train\"].shuffle().map(generate_training_data)\n","    val_data = train_val[\"test\"].shuffle().map(generate_training_data)\n","else:\n","    train_data = data['train'].shuffle().map(generate_training_data)\n","    val_data = None\n","\n","# 使用 Transformers Trainer 進行模型訓練\n","trainer = transformers.Trainer(\n","    model=model,\n","    train_dataset=train_data,\n","    eval_dataset=val_data,\n","    args=transformers.TrainingArguments(\n","        per_device_train_batch_size=MICRO_BATCH_SIZE,\n","        gradient_accumulation_steps=GRADIENT_ACCUMULATION_STEPS,\n","        warmup_steps=50,\n","        num_train_epochs=num_epoch,\n","        learning_rate=LEARNING_RATE,\n","        fp16=True,  # 使用混合精度訓練\n","        logging_steps=logging_steps,\n","        save_strategy=\"steps\",\n","        save_steps=save_steps,\n","        output_dir=ckpt_dir,\n","        save_total_limit=save_total_limit,\n","        ddp_find_unused_parameters=False if ddp else None,  # 是否使用 DDP，控制梯度更新策略\n","        report_to=report_to,\n","    ),\n","    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),\n",")\n","\n","# 禁用模型的 cache 功能\n","model.config.use_cache = False\n","\n","# 若使用 PyTorch 2.0 版本以上且非 Windows 系統，進行模型編譯\n","if torch.__version__ >= \"2\" and sys.platform != 'win32':\n","    model = torch.compile(model)\n","\n","# 開始模型訓練\n","trainer.train()\n","\n","# 將訓練完的模型保存到指定的目錄中\n","model.save_pretrained(ckpt_dir)\n","\n","# 印出訓練過程中可能的缺失權重的警告信息\n","print(\"\\n If there's a warning about missing keys above, please disregard :)\")"]},{"cell_type":"markdown","metadata":{"id":"EKeGb8bRqWux"},"source":["##  Testing\n","The fine-tuning process is done. We then want to test whether our model can do the task that we wanted it to do before but failed."]},{"cell_type":"markdown","metadata":{"id":"v8I3stApnTWb"},"source":["We need to first load the fine-tuned model for checkpoint we saved."]},{"cell_type":"code","execution_count":17,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"5ag6GvOCe9Ql","outputId":"8c8290ae-2e6d-4c33-8470-8f75ca4e8dca","executionInfo":{"status":"ok","timestamp":1738917112935,"user_tz":-480,"elapsed":50,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["all available checkpoints:\n"," id: checkpoint name\n","  0: checkpoint-65\n"]}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","# find all available checkpoints\n","ckpts = []\n","for ckpt in os.listdir(ckpt_dir):\n","    if (ckpt.startswith(\"checkpoint-\")):\n","        ckpts.append(ckpt)\n","\n","# list all the checkpoints\n","ckpts = sorted(ckpts, key = lambda ckpt: int(ckpt.split(\"-\")[-1]))\n","print(\"all available checkpoints:\")\n","print(\" id: checkpoint name\")\n","for (i, ckpt) in enumerate(ckpts):\n","    print(f\"{i:>3}: {ckpt}\")\n"]},{"cell_type":"code","execution_count":18,"metadata":{"id":"khq-LbNlcdfp","executionInfo":{"status":"ok","timestamp":1738917130268,"user_tz":-480,"elapsed":46,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[],"source":["\"\"\" You may want (but not necessarily need) to change the check point \"\"\"\n","\n","id_of_ckpt_to_use = -1  # 要用來進行推理的checkpoint的id(對應上一個cell的輸出結果)\n","                        # 預設值-1指的是上列checkpoints中的\"倒數\"第一個，也就是最後一個checkpoint\n","                        # 如果想要選擇其他checkpoint，可以把-1改成有列出的checkpoint id中的其中一個\n","\n","ckpt_name = os.path.join(ckpt_dir, ckpts[id_of_ckpt_to_use])"]},{"cell_type":"code","execution_count":19,"metadata":{"id":"vdO5i3X3bTus","executionInfo":{"status":"ok","timestamp":1738917147004,"user_tz":-480,"elapsed":2,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[],"source":["\"\"\" You may want (but not necessarily need) to change decoding parameters \"\"\"\n","# 你可以在這裡調整decoding parameter，decoding parameter的詳細解釋請見homework slides\n","max_len = 128   # 生成回復的最大長度\n","temperature = 0.1  # 設定生成回覆的隨機度，值越小生成的回覆越穩定\n","top_p = 0.3  # Top-p (nucleus) 抽樣的機率閾值，用於控制生成回覆的多樣性\n","# top_k = 5 # 調整Top-k值，以增加生成回覆的多樣性和避免生成重複的詞彙"]},{"cell_type":"code","source":["# 增加了下面的代码释放之前占用的显存，防止你遇到 OutOfMemoryError。（如果显存足够大可以注释）\n","import gc\n","\n","# 删除模型和 tokenizer 对象\n","del model\n","del tokenizer\n","\n","# 调用垃圾回收机制，强制释放未使用的内存\n","gc.collect()\n","\n","# 清理 GPU 缓存\n","torch.cuda.empty_cache()"],"metadata":{"id":"oEmO7zkWikdq","executionInfo":{"status":"ok","timestamp":1738917206366,"user_tz":-480,"elapsed":439,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"execution_count":21,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"xTr0DfVBSekD"},"source":["The following code block takes about **2** minutes to run if you use the default setting, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":22,"metadata":{"id":"-wKVPpMVtkql","colab":{"base_uri":"https://localhost:8080/","height":49,"referenced_widgets":["7dd38da0c66841ec8e04136cc0bc55e7","8e0edb8b9f6a4e32af1a09a787a0a4c3","b924800d450a4bf1ae3290a15072c864","82234d0c11cc4ad5986f7803e05aec39","8cf2aab1a02548009a93433ac1064f96","adc4a8adb6c4456982fc7f844303ed49","60ad3f096fc34f83be55954128d6f8d1","5886a9d647c841308d7b935d0359fa2e","9210430c303a491c91d2e093b79ef49c","84674ac6420c43eea2d04dca5b7ed487","a84282cac8c64750bd4e0d658105a6bc"]},"executionInfo":{"status":"ok","timestamp":1738917281790,"user_tz":-480,"elapsed":73483,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"baf30484-e1e7-43a2-9d4b-1110c47824b4"},"outputs":[{"output_type":"display_data","data":{"text/plain":["Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"7dd38da0c66841ec8e04136cc0bc55e7"}},"metadata":{}}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","test_data_path = \"GenAI-Hw5/Tang_testing_data.json\"\n","output_path = os.path.join(output_dir, \"results.txt\")\n","\n","cache_dir = \"./cache\"  # 設定快取目錄路徑\n","seed = 42  # 設定隨機種子，用於重現結果\n","no_repeat_ngram_size = 3  # 設定禁止重複 Ngram 的大小，用於避免生成重複片段\n","\n","nf4_config = BitsAndBytesConfig(\n","   load_in_4bit=True,\n","   bnb_4bit_quant_type=\"nf4\",\n","   bnb_4bit_use_double_quant=True,\n","   bnb_4bit_compute_dtype=torch.bfloat16\n",")\n","\n","# 使用 tokenizer 將模型名稱轉換成模型可讀的數字表示形式\n","tokenizer = AutoTokenizer.from_pretrained(\n","    model_name,\n","    cache_dir=cache_dir,\n","    quantization_config=nf4_config\n",")\n","\n","# 從預訓練模型載入模型並設定為 8 位整數 (INT8) 模型\n","model = AutoModelForCausalLM.from_pretrained(\n","    model_name,\n","    quantization_config=nf4_config,\n","    device_map={'': 0},  # 設定使用的設備，此處指定為 GPU 0\n","    cache_dir=cache_dir\n",")\n","\n","# 從指定的 checkpoint 載入模型權重\n","model = PeftModel.from_pretrained(model, ckpt_name, device_map={'': 0})"]},{"cell_type":"markdown","metadata":{"id":"Vs4_EtnoWA5b"},"source":["The following code block takes about **4** minutes to run if you use the default setting, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":23,"metadata":{"id":"xcHoU2X8xRkn","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1738917340867,"user_tz":-480,"elapsed":46037,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"7ddfcd4d-fcaa-4279-a164-c81fe8f99505"},"outputs":[{"output_type":"stream","name":"stdout","text":["1. 雪霽銀妝素，桔高映瓊枝。風起玉露落，月明金殿空。\n","2. 夫子何爲者？栖栖一代中。不學禮樂者，不學兵法者。\n","3. 飛蓋去芳園，蘭橈遊翠渚。風起水波動，月落山色空。\n","4. 條風開獻節，灰律動初陽。春色滿園裏，春聲入宮中。\n","5. 昨夜星辰昨夜風，畫樓西畔桂堂東。今朝朝露今朝霜，玉關寒夜寒中。\n","6. 三日入廚下，洗手作羹湯。香煙升殿上，玉盤映玉盤。\n","7. 嵩雲秦樹久離居，雙鯉迢迢一紙書。書中言情難相傳，風中聲色更相許。\n","8. 慨然撫長劒，濟世豈邀名。今朝不復見，何日復相迎。\n","9. 乘興南遊不戒嚴，九重誰省諫書函。春色滿園春色盡，花落春色不復見。\n","10. 猿鳥猶疑畏簡書，風雲常爲護儲胥。爼爿未開先有功，爯爻未成先有德。\n","11. 君問歸期未有期，巴山夜雨漲秋池。月明孤舟獨不歸，風起孤花落無時。\n","12. 相見時難別亦難，東風無力百花殘。年華已去不復返，春色盡盡不復見。\n","13. 雲母屏風燭影深，長河漸落曉星沈。孤燈不滅夜未央，孤心不散夢未醒。\n","14. 高閣客竟去，小園花亂飛。風起落花滿，月落落花死。\n","15. 瑤池阿母綺窗開，黃竹歌聲動地哀。玉露滿地無煙露，金風吹動玉樹花。\n"]}],"source":["\"\"\" It is recommmended NOT to change codes in this cell \"\"\"\n","\n","results = []\n","\n","# 設定生成配置，包括隨機度、束搜索等相關參數\n","generation_config = GenerationConfig(\n","    do_sample=True,\n","    temperature=temperature,\n","    num_beams=1,\n","    top_p=top_p,\n","    # top_k=top_k,\n","    no_repeat_ngram_size=no_repeat_ngram_size,\n","    pad_token_id=2\n",")\n","\n","# 讀取測試資料\n","with open(test_data_path, \"r\", encoding = \"utf-8\") as f:\n","    test_datas = json.load(f)\n","\n","# 對於每個測試資料進行預測，並存下結果\n","with open(output_path, \"w\", encoding = \"utf-8\") as f:\n","  for (i, test_data) in enumerate(test_datas):\n","      predict = evaluate(test_data[\"instruction\"], generation_config, max_len, test_data[\"input\"], verbose = False)\n","      f.write(f\"{i+1}. \"+test_data[\"input\"]+predict+\"\\n\")\n","      print(f\"{i+1}. \"+test_data[\"input\"]+predict)\n"]},{"cell_type":"markdown","source":["## **IMPORTANT**: Submit above 15 poems to DaVinci Assistant.\n","You can find these poems in your \"results.txt\". The grading of this homework will be based on the evaluation results of DaVinci Assistant on these poems."],"metadata":{"id":"1tJT1WINKXV7"}},{"cell_type":"markdown","source":["## See how the fine-tune model do compared to model without fine-tuning"],"metadata":{"id":"sT78LeWaJM4D"}},{"cell_type":"markdown","metadata":{"id":"tx_mZZAZWdiE"},"source":["We now check what our model can do on the same examples we saw in the \"Inference before Fine-tuning\" section above.\n","\n","The following code block takes about **40** seconds to run if you use the default setting, but it may vary depending on the condition of Colab."]},{"cell_type":"code","execution_count":24,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"BtDG2WhIWZlS","outputId":"ae75ecbc-348b-4de4-db9b-1ffd62afd1d2","executionInfo":{"status":"ok","timestamp":1738917355727,"user_tz":-480,"elapsed":10045,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["Example 1:\n","模型輸入:\n","以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。相見時難別亦難，東風無力百花殘。\n","\n","模型輸出:\n","年華已去不復返，春色盡盡不復見。\n","--------------------------------------------------------------------------------\n","Example 2:\n","模型輸入:\n","以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。重帷深下莫愁堂，臥後清宵細細長。\n","\n","模型輸出:\n","月色入戶照玉樓，風聲起樓動金管。\n","--------------------------------------------------------------------------------\n","Example 3:\n","模型輸入:\n","以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。芳辰追逸趣，禁苑信多奇。\n","\n","模型輸出:\n","春色映翠樹，花影落紅地。\n","--------------------------------------------------------------------------------\n"]}],"source":["# using the same demo examples as before\n","test_tang_list = ['相見時難別亦難，東風無力百花殘。', '重帷深下莫愁堂，臥後清宵細細長。', '芳辰追逸趣，禁苑信多奇。']\n","\n","# inference our fine-tuned model\n","demo_after_finetune = []\n","for tang in test_tang_list:\n","  demo_after_finetune.append(f'模型輸入:\\n以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。{tang}\\n\\n模型輸出:\\n'+evaluate('以下是一首唐詩的第一句話，請用你的知識判斷並完成整首詩。', generation_config, max_len, tang, verbose = False))\n","\n","# print and store the output to text file\n","for idx in range(len(demo_after_finetune)):\n","  print(f\"Example {idx + 1}:\")\n","  print(demo_after_finetune[idx])\n","  print(\"-\" * 80)\n"]},{"cell_type":"markdown","source":["## **IMPORTANT**: DO NOT submit the above 3 examples to DaVinci Assistant.\n","This 3 examples are only used for comparing how model peforms before and after fine-tuning\n"],"metadata":{"id":"wagFeDX4Hpa8"}},{"cell_type":"markdown","metadata":{"id":"OWgTS13bgnWQ"},"source":["## Download Results\n","You MUST have this file to finish your homework. If your browser does not download it automatically, you can find it in your Google Drive."]},{"cell_type":"code","execution_count":25,"metadata":{"id":"rUDuNQOY4os_","colab":{"base_uri":"https://localhost:8080/","height":17},"executionInfo":{"status":"ok","timestamp":1738917430030,"user_tz":-480,"elapsed":35,"user":{"displayName":"Jacky hu","userId":"02919603212216756843"}},"outputId":"cbb1db58-61f1-4986-cc3a-da12261fcc7a"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["\n","    async function download(id, filename, size) {\n","      if (!google.colab.kernel.accessAllowed) {\n","        return;\n","      }\n","      const div = document.createElement('div');\n","      const label = document.createElement('label');\n","      label.textContent = `Downloading \"${filename}\": `;\n","      div.appendChild(label);\n","      const progress = document.createElement('progress');\n","      progress.max = size;\n","      div.appendChild(progress);\n","      document.body.appendChild(div);\n","\n","      const buffers = [];\n","      let downloaded = 0;\n","\n","      const channel = await google.colab.kernel.comms.open(id);\n","      // Send a message to notify the kernel that we're ready.\n","      channel.send({})\n","\n","      for await (const message of channel.messages) {\n","        // Send a message to notify the kernel that we're ready.\n","        channel.send({})\n","        if (message.buffers) {\n","          for (const buffer of message.buffers) {\n","            buffers.push(buffer);\n","            downloaded += buffer.byteLength;\n","            progress.value = downloaded;\n","          }\n","        }\n","      }\n","      const blob = new Blob(buffers, {type: 'application/binary'});\n","      const a = document.createElement('a');\n","      a.href = window.URL.createObjectURL(blob);\n","      a.download = filename;\n","      div.appendChild(a);\n","      a.click();\n","      div.remove();\n","    }\n","  "]},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.Javascript object>"],"application/javascript":["download(\"download_716c7545-e845-428d-9f11-f192a150105d\", \"results.txt\", 1335)"]},"metadata":{}}],"source":["from google.colab import files\n","files.download(output_path)"]},{"cell_type":"markdown","source":["## Reference\n","\n","[Tang Poem Dataset](https://github.com/chinese-poetry/chinese-poetry/tree/master/%E5%85%A8%E5%94%90%E8%AF%97?fbclid=IwAR2bM14S42T-VtrvMi3wywCqKfYJraBtMl7QVTo0qyPMjX9jj9Vj3JepFBA)"],"metadata":{"id":"PpkNFzvLz7iI"}}],"metadata":{"accelerator":"GPU","colab":{"gpuType":"T4","provenance":[{"file_id":"1nB3jwRJVKXSDDNO-pbURrao0N2MpqHl8","timestamp":1738746164594}]},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"},"widgets":{"application/vnd.jupyter.widget-state+json":{"bcfd3dc154284e4b965b8bc67ff78cec":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_29bbe0b236f84b2cba1855d7588e79d1","IPY_MODEL_24e7258b92454dc9827d2957d5fca17e","IPY_MODEL_49139cb882484ef2a4ab90cd3ad8b57d"],"layout":"IPY_MODEL_0891c210b7c94702a9b59a8405d238f0"}},"29bbe0b236f84b2cba1855d7588e79d1":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_55fdaa9a899f4d4393b6165d0e46ff1e","placeholder":"​","style":"IPY_MODEL_c654b9c398124d63a45f618113f05217","value":"config.json: 100%"}},"24e7258b92454dc9827d2957d5fca17e":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8b198efb30f8497194b01f000097b492","max":618,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6a4154e2b36241e98796d55f8e95f032","value":618}},"49139cb882484ef2a4ab90cd3ad8b57d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4029dc88032d4c6f9a71acf1da7d8da2","placeholder":"​","style":"IPY_MODEL_a70623b2fdaa4266b636262801e11d2c","value":" 618/618 [00:00&lt;00:00, 45.0kB/s]"}},"0891c210b7c94702a9b59a8405d238f0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"55fdaa9a899f4d4393b6165d0e46ff1e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c654b9c398124d63a45f618113f05217":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8b198efb30f8497194b01f000097b492":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a4154e2b36241e98796d55f8e95f032":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"4029dc88032d4c6f9a71acf1da7d8da2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a70623b2fdaa4266b636262801e11d2c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"820b125bec9044b5a33378e0269261ec":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_23b0b3904ecc4220844926e574ab112d","IPY_MODEL_1096aa12633b4bd08dbe83932cc8a757","IPY_MODEL_f707fdf45ba846fab2f912d082d536b1"],"layout":"IPY_MODEL_109991a5f4464902bf868d3921cdc2b9"}},"23b0b3904ecc4220844926e574ab112d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2d4c62aacff84ef38c962fe528dfca03","placeholder":"​","style":"IPY_MODEL_cfce5e6ff1a54e9081201ae6d1abe90b","value":"model.safetensors.index.json: 100%"}},"1096aa12633b4bd08dbe83932cc8a757":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_82186e29d0704b98a62f67f08c55e862","max":25125,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6911fda9a1434b8697b0840a01dbe4b0","value":25125}},"f707fdf45ba846fab2f912d082d536b1":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_82b4f6a98ace423a9fee5183293a39e5","placeholder":"​","style":"IPY_MODEL_340932d9058d4897a41026023402b8e1","value":" 25.1k/25.1k [00:00&lt;00:00, 1.78MB/s]"}},"109991a5f4464902bf868d3921cdc2b9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2d4c62aacff84ef38c962fe528dfca03":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cfce5e6ff1a54e9081201ae6d1abe90b":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"82186e29d0704b98a62f67f08c55e862":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6911fda9a1434b8697b0840a01dbe4b0":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"82b4f6a98ace423a9fee5183293a39e5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"340932d9058d4897a41026023402b8e1":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"bce2c017697d4c2480d28ab758852d2c":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_cacab48531b24d549da3c76e1f526687","IPY_MODEL_d7cb2badde8548e394127b5554dc24d9","IPY_MODEL_4df6ff5f4d094ce29d00047ef0777635"],"layout":"IPY_MODEL_cf0d138e965a4bc8b2dc66affda9c98c"}},"cacab48531b24d549da3c76e1f526687":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_0edbdd84974242a3b7887b98f8457b9a","placeholder":"​","style":"IPY_MODEL_c0a47a3d93db4a97ab179ad4069a831f","value":"Downloading shards: 100%"}},"d7cb2badde8548e394127b5554dc24d9":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_46d034094cbd4484a1664e2ad6877457","max":2,"min":0,"orientation":"horizontal","style":"IPY_MODEL_3dd9c00a6b32462bb4653357f9fc5eb7","value":2}},"4df6ff5f4d094ce29d00047ef0777635":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3819b9d87fa147be83e1c7b64d8418ca","placeholder":"​","style":"IPY_MODEL_c1ef0cddbcec45f58fedd26807a03567","value":" 2/2 [05:57&lt;00:00, 168.00s/it]"}},"cf0d138e965a4bc8b2dc66affda9c98c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0edbdd84974242a3b7887b98f8457b9a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c0a47a3d93db4a97ab179ad4069a831f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"46d034094cbd4484a1664e2ad6877457":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3dd9c00a6b32462bb4653357f9fc5eb7":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"3819b9d87fa147be83e1c7b64d8418ca":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c1ef0cddbcec45f58fedd26807a03567":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"666066284fc04acfb3f1ffc66f78cc1d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_28007f024c5742aabb0a9bc1f6640662","IPY_MODEL_d1de74fcdbff45dda9c5024d63df87b6","IPY_MODEL_4305114af98e45a9ae9c8d88866ea7c7"],"layout":"IPY_MODEL_3240c32f8b40481a8150d3b7fb05cd95"}},"28007f024c5742aabb0a9bc1f6640662":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c8a87b0c75a14436887d927df61409a4","placeholder":"​","style":"IPY_MODEL_14a84072c51243ceb1162ee19a796ef3","value":"model-00001-of-00002.safetensors: 100%"}},"d1de74fcdbff45dda9c5024d63df87b6":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c6b5cea33d5c40fc975ab835a7ef83bc","max":9987004688,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e29de32052a24575a107c9f57df2fca5","value":9987004688}},"4305114af98e45a9ae9c8d88866ea7c7":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_004df69a0ab247649ba99ef586f20741","placeholder":"​","style":"IPY_MODEL_1ae50934a52849da845a77da7eb3e4e2","value":" 9.99G/9.99G [03:57&lt;00:00, 41.8MB/s]"}},"3240c32f8b40481a8150d3b7fb05cd95":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c8a87b0c75a14436887d927df61409a4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"14a84072c51243ceb1162ee19a796ef3":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c6b5cea33d5c40fc975ab835a7ef83bc":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e29de32052a24575a107c9f57df2fca5":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"004df69a0ab247649ba99ef586f20741":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1ae50934a52849da845a77da7eb3e4e2":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"76bd16da496c4f57b85c6a369869fa3b":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_c03520becbf5402fb1e6e2aab6bad158","IPY_MODEL_0d27882a68bc468c946c61e803f12001","IPY_MODEL_ee39c60ad86d4d50b27656ddd63bfd22"],"layout":"IPY_MODEL_140b00a74bea4582a20e407107da6251"}},"c03520becbf5402fb1e6e2aab6bad158":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d401a7e1f4b8427f85ff2ad1f85f4143","placeholder":"​","style":"IPY_MODEL_55d6e9b0d1af439abc98002f45b1c8c1","value":"model-00002-of-00002.safetensors: 100%"}},"0d27882a68bc468c946c61e803f12001":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a4cd4d55813343fa90567018063023ae","max":4987226928,"min":0,"orientation":"horizontal","style":"IPY_MODEL_2e36e1bd3b8c49ee9f050ac5cc04397f","value":4987226928}},"ee39c60ad86d4d50b27656ddd63bfd22":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8eb769686d24421185b02eb627d3788e","placeholder":"​","style":"IPY_MODEL_a9e9a511989f41d1bef73532f67d1457","value":" 4.99G/4.99G [01:58&lt;00:00, 43.5MB/s]"}},"140b00a74bea4582a20e407107da6251":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d401a7e1f4b8427f85ff2ad1f85f4143":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"55d6e9b0d1af439abc98002f45b1c8c1":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a4cd4d55813343fa90567018063023ae":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2e36e1bd3b8c49ee9f050ac5cc04397f":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"8eb769686d24421185b02eb627d3788e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a9e9a511989f41d1bef73532f67d1457":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3c5c70522e77433796cefc9e2c13953e":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_3f1e3a686adb401288f09689de544a3b","IPY_MODEL_ebaaac57fac34b2293cad1e4bc5da112","IPY_MODEL_b3cc0a7bfbb74d5489a2b5352df702dc"],"layout":"IPY_MODEL_73dc368955864dbe8a704188c06bbc4b"}},"3f1e3a686adb401288f09689de544a3b":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e42360c06b424aa9badab184d62dd451","placeholder":"​","style":"IPY_MODEL_7668843d095943ad862509f0cc690bc7","value":"Loading checkpoint shards: 100%"}},"ebaaac57fac34b2293cad1e4bc5da112":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_addf29833b5d4b7790a688a42a073d50","max":2,"min":0,"orientation":"horizontal","style":"IPY_MODEL_478e77ba0f9c4b2da2c847440df7105c","value":2}},"b3cc0a7bfbb74d5489a2b5352df702dc":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_623e941b653d48f9bf8ed1fe3e17fb12","placeholder":"​","style":"IPY_MODEL_cdf9c8d0abf34e26b83346fe5d844ce5","value":" 2/2 [01:10&lt;00:00, 33.09s/it]"}},"73dc368955864dbe8a704188c06bbc4b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e42360c06b424aa9badab184d62dd451":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7668843d095943ad862509f0cc690bc7":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"addf29833b5d4b7790a688a42a073d50":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"478e77ba0f9c4b2da2c847440df7105c":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"623e941b653d48f9bf8ed1fe3e17fb12":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cdf9c8d0abf34e26b83346fe5d844ce5":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f238d91216ef49808d854faa5a3de369":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_ec1e6656ee174169ac12b33bca02a8e5","IPY_MODEL_2b02c83f74a24bcfbc241f0e4eede897","IPY_MODEL_f627283c91704f7295f342ea87b6c018"],"layout":"IPY_MODEL_0a0120ff6ad14d0f9927d9e9162a9afc"}},"ec1e6656ee174169ac12b33bca02a8e5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6351ee9201104df7b594c30f1813dc8e","placeholder":"​","style":"IPY_MODEL_68c7d5856c3544f29d5a79870fef6d40","value":"generation_config.json: 100%"}},"2b02c83f74a24bcfbc241f0e4eede897":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_21074198381245af9fe93a519946b411","max":111,"min":0,"orientation":"horizontal","style":"IPY_MODEL_22b961876ded4e638d600661973fcf4d","value":111}},"f627283c91704f7295f342ea87b6c018":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f784ff58fc4c4d9a963296fa0bab036b","placeholder":"​","style":"IPY_MODEL_1823b59a47264480b5e897e789930b42","value":" 111/111 [00:00&lt;00:00, 7.48kB/s]"}},"0a0120ff6ad14d0f9927d9e9162a9afc":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6351ee9201104df7b594c30f1813dc8e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"68c7d5856c3544f29d5a79870fef6d40":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"21074198381245af9fe93a519946b411":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"22b961876ded4e638d600661973fcf4d":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f784ff58fc4c4d9a963296fa0bab036b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1823b59a47264480b5e897e789930b42":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"99d2dde3fcf345118b1a9731c5ccd04d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d36b4697da1d4f61b9e30a7cf3aaf0e9","IPY_MODEL_f0f72eb4c633433b9c2f6a5b4183f497","IPY_MODEL_aad2a2577b624a0480467cf110230162"],"layout":"IPY_MODEL_2d9ac1a648a34f7b8f94b444b2a04ffd"}},"d36b4697da1d4f61b9e30a7cf3aaf0e9":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7f8c08bf7693425dbea463c363ce5973","placeholder":"​","style":"IPY_MODEL_e0ef488369ea4e7d993e3ff46816225e","value":"tokenizer_config.json: 100%"}},"f0f72eb4c633433b9c2f6a5b4183f497":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e772972c082a45bba49dadc7f6296481","max":2293,"min":0,"orientation":"horizontal","style":"IPY_MODEL_8b103e043b6d43cf917b244b625d05b6","value":2293}},"aad2a2577b624a0480467cf110230162":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1bd01d619b1e400d96f5ffdc89c6c72d","placeholder":"​","style":"IPY_MODEL_27155d4f1cb4473c954096455d7ee7b6","value":" 2.29k/2.29k [00:00&lt;00:00, 134kB/s]"}},"2d9ac1a648a34f7b8f94b444b2a04ffd":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7f8c08bf7693425dbea463c363ce5973":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e0ef488369ea4e7d993e3ff46816225e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e772972c082a45bba49dadc7f6296481":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8b103e043b6d43cf917b244b625d05b6":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1bd01d619b1e400d96f5ffdc89c6c72d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"27155d4f1cb4473c954096455d7ee7b6":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"52bf15bf4d4f47e4acad99cd0457dd35":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1900cba726d64d3fb527191fe4e22b2f","IPY_MODEL_74d8c2d0f13e4388bb188f0803db840f","IPY_MODEL_428cfa7372af4f8ca4ce33303a0847c8"],"layout":"IPY_MODEL_d6fb6a4af4ce4b55adf75e9eaf31aef3"}},"1900cba726d64d3fb527191fe4e22b2f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_557f2b17f9fc49ad89cc15474976b99e","placeholder":"​","style":"IPY_MODEL_d50e8cbc367c46a293907b36f0b3732c","value":"tokenizer.model: 100%"}},"74d8c2d0f13e4388bb188f0803db840f":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_6e5a02789b3f47949654e26890e22422","max":911034,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9f19625acd984d6d912a92f1a91b3c89","value":911034}},"428cfa7372af4f8ca4ce33303a0847c8":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_af2ff52b55ec42daa8116b5d4d02a672","placeholder":"​","style":"IPY_MODEL_f6c0fb2e19534f3287851932defaa4b3","value":" 911k/911k [00:00&lt;00:00, 4.06MB/s]"}},"d6fb6a4af4ce4b55adf75e9eaf31aef3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"557f2b17f9fc49ad89cc15474976b99e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d50e8cbc367c46a293907b36f0b3732c":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6e5a02789b3f47949654e26890e22422":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9f19625acd984d6d912a92f1a91b3c89":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"af2ff52b55ec42daa8116b5d4d02a672":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f6c0fb2e19534f3287851932defaa4b3":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"04d073d67706463ca88a81fbe87e77f0":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_7c8f02283f4149e5bb58b8ad16d93ac4","IPY_MODEL_42d4c57ffc3b4d5a84b1971cec46cdf9","IPY_MODEL_9ed625ec2d5345a98639b040d84dfa2b"],"layout":"IPY_MODEL_e0793957ac1047538ce377c84bea1674"}},"7c8f02283f4149e5bb58b8ad16d93ac4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_927530148342418d88c2793afd4b5e46","placeholder":"​","style":"IPY_MODEL_6687d3d94327447cb8f8cc85dd8b63d8","value":"tokenizer.json: 100%"}},"42d4c57ffc3b4d5a84b1971cec46cdf9":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_479428fa05214e4c9dc6a86088fbee12","max":2786201,"min":0,"orientation":"horizontal","style":"IPY_MODEL_526470a79ce248c8861201ca0fb6e6e5","value":2786201}},"9ed625ec2d5345a98639b040d84dfa2b":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7758977be2cf47b4869c24fa8ae59b4e","placeholder":"​","style":"IPY_MODEL_ce2cc8aeeece4bd59a84d730f962321f","value":" 2.79M/2.79M [00:00&lt;00:00, 13.5MB/s]"}},"e0793957ac1047538ce377c84bea1674":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"927530148342418d88c2793afd4b5e46":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6687d3d94327447cb8f8cc85dd8b63d8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"479428fa05214e4c9dc6a86088fbee12":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"526470a79ce248c8861201ca0fb6e6e5":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"7758977be2cf47b4869c24fa8ae59b4e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ce2cc8aeeece4bd59a84d730f962321f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"eb5c8ceab7984b05a14337dfdb4be86d":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_fa77618315274b93bbc35c99e61ff262","IPY_MODEL_5cd0801333334acd963d3831745317dc","IPY_MODEL_51f2b2ec63674da9ad545ae353db20f2"],"layout":"IPY_MODEL_b8b416066fbe48a3a0cd47062fdee18f"}},"fa77618315274b93bbc35c99e61ff262":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c5f2e03f86244d74829ca03d0cbaeabb","placeholder":"​","style":"IPY_MODEL_8a7ff402838e44929b2d07f88f88226a","value":"added_tokens.json: 100%"}},"5cd0801333334acd963d3831745317dc":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_f4f8559486bf49ac865e1c89b23fac96","max":39,"min":0,"orientation":"horizontal","style":"IPY_MODEL_1bf1eac05a3b48a49f6c01ca2471dd68","value":39}},"51f2b2ec63674da9ad545ae353db20f2":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2738c81c8c334e28b126843c5b6a9c2f","placeholder":"​","style":"IPY_MODEL_828d2fa937f742638757378393ae9557","value":" 39.0/39.0 [00:00&lt;00:00, 2.71kB/s]"}},"b8b416066fbe48a3a0cd47062fdee18f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c5f2e03f86244d74829ca03d0cbaeabb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8a7ff402838e44929b2d07f88f88226a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f4f8559486bf49ac865e1c89b23fac96":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1bf1eac05a3b48a49f6c01ca2471dd68":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"2738c81c8c334e28b126843c5b6a9c2f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"828d2fa937f742638757378393ae9557":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"34015942bc704bdcb4076d2132fcc0f6":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d704822cb44940b89fc1f1054969ec17","IPY_MODEL_c5d30f3ea5f44171855ffed11ccd3d12","IPY_MODEL_85701f001be740e5b0c3390a375caa0e"],"layout":"IPY_MODEL_e6b9f589d4a8492a9ad4e2967afe5b33"}},"d704822cb44940b89fc1f1054969ec17":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5748406df7434a6e98e2e48d23a918de","placeholder":"​","style":"IPY_MODEL_40a72a290f9540048425f78637775a91","value":"special_tokens_map.json: 100%"}},"c5d30f3ea5f44171855ffed11ccd3d12":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_e62909803e184e938879d1fa77ff7fab","max":551,"min":0,"orientation":"horizontal","style":"IPY_MODEL_b70acf67d7da46f2b6d32b72b31c7d87","value":551}},"85701f001be740e5b0c3390a375caa0e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1587793d20e94d3a83f2f2db74fe91a4","placeholder":"​","style":"IPY_MODEL_f8a3582a5ee64fa291592fee5e6a8a41","value":" 551/551 [00:00&lt;00:00, 37.9kB/s]"}},"e6b9f589d4a8492a9ad4e2967afe5b33":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5748406df7434a6e98e2e48d23a918de":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"40a72a290f9540048425f78637775a91":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"e62909803e184e938879d1fa77ff7fab":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b70acf67d7da46f2b6d32b72b31c7d87":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1587793d20e94d3a83f2f2db74fe91a4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f8a3582a5ee64fa291592fee5e6a8a41":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"701997d9abf6402299b355c8a3fe8623":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_956ae37532b241a188a9aba7f87b7d46","IPY_MODEL_3827589f4d68440f8e164fc234a6b0be","IPY_MODEL_6dfc9dc0c0e244afb7690d3f8a34c10e"],"layout":"IPY_MODEL_ce798caee48846099b6e9086ffdbb104"}},"956ae37532b241a188a9aba7f87b7d46":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ce7f76813e24443ebcd18a7cf828d95b","placeholder":"​","style":"IPY_MODEL_7cddd41aa9de482b995f3355a30e8d20","value":"Generating train split: "}},"3827589f4d68440f8e164fc234a6b0be":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_1e49c31e57f8427c882f1d4435c497ff","max":1,"min":0,"orientation":"horizontal","style":"IPY_MODEL_abf06b78f9b5419a99007a3b755a99bc","value":1}},"6dfc9dc0c0e244afb7690d3f8a34c10e":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f5800fd73dd84b46b62ad12f64f01824","placeholder":"​","style":"IPY_MODEL_137854b9b7334959863b836b071e0e98","value":" 1040/0 [00:00&lt;00:00, 7525.15 examples/s]"}},"ce798caee48846099b6e9086ffdbb104":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ce7f76813e24443ebcd18a7cf828d95b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7cddd41aa9de482b995f3355a30e8d20":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1e49c31e57f8427c882f1d4435c497ff":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":"20px"}},"abf06b78f9b5419a99007a3b755a99bc":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f5800fd73dd84b46b62ad12f64f01824":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"137854b9b7334959863b836b071e0e98":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"f5d4abbaec95435d9d52f452b3465b39":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_abe48faf2fe44cd1a1844d27e1e81fde","IPY_MODEL_003cec5011584bf38d4eb4670c57c0e4","IPY_MODEL_18c7781c59064d1ebed245cab21c256f"],"layout":"IPY_MODEL_cb3dd64ff5cb4bdf98c333f27614f843"}},"abe48faf2fe44cd1a1844d27e1e81fde":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_d638472d17c44b258a1572ce965ceec1","placeholder":"​","style":"IPY_MODEL_2dce2f8cf3ca4d3fabb559da4aa52bad","value":"Map: 100%"}},"003cec5011584bf38d4eb4670c57c0e4":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8bcef1ca5bc44d98bf9ee1518f4a8d3f","max":1040,"min":0,"orientation":"horizontal","style":"IPY_MODEL_73b914a2990b46d2aa207fdd7c1aceb8","value":1040}},"18c7781c59064d1ebed245cab21c256f":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_22f94dfc5ab4461fa43c5b6f54be7be9","placeholder":"​","style":"IPY_MODEL_f68ac26de2404b43b3f11e4055345780","value":" 1040/1040 [00:01&lt;00:00, 812.20 examples/s]"}},"cb3dd64ff5cb4bdf98c333f27614f843":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d638472d17c44b258a1572ce965ceec1":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2dce2f8cf3ca4d3fabb559da4aa52bad":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8bcef1ca5bc44d98bf9ee1518f4a8d3f":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"73b914a2990b46d2aa207fdd7c1aceb8":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"22f94dfc5ab4461fa43c5b6f54be7be9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"f68ac26de2404b43b3f11e4055345780":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"53a85e86099045978c28eee131501e43":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_adb03ec94e334ab69e320168cdd8673c","IPY_MODEL_bff036d428c74bf09f7953d65a5ecd16","IPY_MODEL_78eed1ace80746e7b9cf06e75c4e56d4"],"layout":"IPY_MODEL_b661478f72734a649db8779b04dc84a0"}},"adb03ec94e334ab69e320168cdd8673c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_bcdfb7a75f5d454985ef2ed43590b0f6","placeholder":"​","style":"IPY_MODEL_3845bcd464b84a9885eabae4e3b06b76","value":"config.json: 100%"}},"bff036d428c74bf09f7953d65a5ecd16":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_536b0eaa47cb4b6084eba8058da85014","max":618,"min":0,"orientation":"horizontal","style":"IPY_MODEL_77e8d7ee81134623829b50faa88b6150","value":618}},"78eed1ace80746e7b9cf06e75c4e56d4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b7bfa561b03d44f1b2258f183550f00a","placeholder":"​","style":"IPY_MODEL_0de6e2af6120413dacb838d007611c96","value":" 618/618 [00:00&lt;00:00, 35.0kB/s]"}},"b661478f72734a649db8779b04dc84a0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bcdfb7a75f5d454985ef2ed43590b0f6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3845bcd464b84a9885eabae4e3b06b76":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"536b0eaa47cb4b6084eba8058da85014":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"77e8d7ee81134623829b50faa88b6150":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b7bfa561b03d44f1b2258f183550f00a":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0de6e2af6120413dacb838d007611c96":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"7dd38da0c66841ec8e04136cc0bc55e7":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8e0edb8b9f6a4e32af1a09a787a0a4c3","IPY_MODEL_b924800d450a4bf1ae3290a15072c864","IPY_MODEL_82234d0c11cc4ad5986f7803e05aec39"],"layout":"IPY_MODEL_8cf2aab1a02548009a93433ac1064f96"}},"8e0edb8b9f6a4e32af1a09a787a0a4c3":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_adc4a8adb6c4456982fc7f844303ed49","placeholder":"​","style":"IPY_MODEL_60ad3f096fc34f83be55954128d6f8d1","value":"Loading checkpoint shards: 100%"}},"b924800d450a4bf1ae3290a15072c864":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_5886a9d647c841308d7b935d0359fa2e","max":2,"min":0,"orientation":"horizontal","style":"IPY_MODEL_9210430c303a491c91d2e093b79ef49c","value":2}},"82234d0c11cc4ad5986f7803e05aec39":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_84674ac6420c43eea2d04dca5b7ed487","placeholder":"​","style":"IPY_MODEL_a84282cac8c64750bd4e0d658105a6bc","value":" 2/2 [01:11&lt;00:00, 33.45s/it]"}},"8cf2aab1a02548009a93433ac1064f96":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"adc4a8adb6c4456982fc7f844303ed49":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"60ad3f096fc34f83be55954128d6f8d1":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5886a9d647c841308d7b935d0359fa2e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9210430c303a491c91d2e093b79ef49c":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"84674ac6420c43eea2d04dca5b7ed487":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a84282cac8c64750bd4e0d658105a6bc":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":0}